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ABSTRACT 

This article is the second in a series in which we perform an extensive comparison of vari¬ 
ous galaxy-based cluster mass estimation techniques that utilise the positions, velocities and 
colours of galaxies. Our aim is to quantify the scatter, systematic bias and completeness of 
cluster masses derived from a diverse set of 25 galaxy-based methods using two contrasting 
mock galaxy catalogues based on a sophisticated halo occupation model and a semi-analytic 
model. Analysing 968 clusters, we find a wide range in the RMS errors in log M 200 C deliv¬ 
ered by the different methods (0.18 to 1.08 dex, i.e., a factor of ~1.5 to 12), with abundance 
matching and richness methods providing the best results, irrespective of the input model as¬ 
sumptions. In addition, certain methods produce a significant number of catastrophic cases 
where the mass is under- or over-estimated by a factor greater than 10. Given the steeply 
falling high-mass end of the cluster mass function, we recommend that richness or abundance 
matching-based methods are used in conjunction with these methods as a sanity check for 
studies selecting high mass clusters. We see a stronger correlation of the recovered to input 
number of galaxies for both catalogues in comparison with the group/cluster mass, however, 
this does not guarantee that the correct member galaxies are being selected. We do not observe 
significantly higher scatter for either mock galaxy catalogues. Our results have implications 
for cosmological analyses that utilise the masses, richnesses, or abundances of clusters, which 
have different uncertainties when different methods are used. 

Key words: galaxies: clusters - cosmology: observations - galaxies: haloes - galaxies: kine¬ 
matics and dynamics - methods: numerical - methods: statistical 
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1 INTRODUCTION 

Statistical studies of the galaxy cluster population, in particular the 
cluster mass function, provide indispensable knowledge of cosmo¬ 
logical model parameters (see Allen et al. 2011 for a review. Tin¬ 
ker et al. 2012), large scale structure (e.g., Bahcall 1988; Einasto 
et al. 2001; Yang et al. 2005b; Papovich 2008; Willis et al. 2013) 
and galaxy evolution (e.g.. Goto et al. 2003; Postman et al. 2005; 
Martinez et al. 2008). However, deducing accurate masses of these 
gravitationally bound structures remains a fundamental challenge 
for current and future cosmological studies. 

A variety of techniques exist to detect galaxy clusters, and 
from this their masses can be estimated in a number of different 
ways. However, cluster masses cannot be directly measured, but 
only indirectly inferred from observed properties that are correlated 
with mass. To maximise the constraining power of clusters for fu¬ 
ture cosmological surveys, it is essential to characterise the level 
of scatter and systematic bias associated with these mass proxies. 
The Galaxy Cluster Mass Reconstruction Project was created in or¬ 
der to ascertain how accurately we can measure cluster masses us¬ 
ing techniques that rely upon the positions, velocities, colours and 
magnitudes of galaxies. Our goals are to quantify the systematic 
bias, intrinsic scatter and completeness that these methods produce 
and try to enhance their performance by deducing which type of 
method (or combination of methods) is best for any given observa¬ 
tional set-up. 

There are three general steps that galaxy-based techniques fol¬ 
low. The first is to locate the cluster overdensity and determine the 
cluster centre, the second is to choose which galaxies are mem¬ 
bers of the cluster and the final step is to use the properties of 
this membership to estimate a cluster mass. Popular cluster finding 
techniques include using Red Sequence filtering techniques (e.g., 
Gladders & Yee 2000; Murphy et al. 2012; Rykoff et al. 2014) and 
brightest cluster galaxy (BCG) searches (e.g., Yang et al. 2005a; 
Koester et al. 2007). Friends-Of-Friends (FOF) group-finding al¬ 
gorithm based methods are also widely used (e.g., Berlind et al. 
2006; Li & Yee 2008; Jian et al. 2014; Tempel et al. 2014, see FOF 
optimisation study of Duarte & Mamon, 2014), along with methods 
based upon Voronoi tessellation (e.g., Marinoni et al. 2002; Lopes 
et al. 2004; van Breukelen & Clewley 2009; Soares-Santos et al. 
2011). Finally, the magnitudes and positions of galaxies are also 
used to search for over-densities via the matched filter algorithm 
(e.g.. Postman et al. 1996; Olsen et al. 1999; Kepner et al. 1999; 
Menanteau et al. 2009). 

The second procedure of galaxy-based mass estimation is to 
deduce accurate galaxy membership. Initial membership can be 
chosen in a variety of ways. Some methods use the galaxies ob¬ 
tained during the first step of the cluster overdensity search via the 
FOF algorithm (e.g., Yang et al. 2005b; Yang et al. 2007; Munoz- 
Cuartas & Muller 2012; Tempel et al. 2014; Pearson et al. in prepa¬ 
ration). Other commonly used methods are to select galaxies within 
a specified region of the colour-magnitude space (e.g., Saro et al. 
2013) or in projected phase space (e.g., von der Linden et al. 2007; 
Wojtak et al. 2009; Mamon et al. 2013; Gifford & Miller 2013; 
Sifon et al. 2013; Pearson et al. in preparation). Though these tech¬ 
niques generate an impression of which galaxies are associated 
with a cluster, deducing which galaxies are true members of the 
cluster is often problematic due to interloping galaxies. These in¬ 
terlopers are close to but not gravitationally bound to the cluster 
and their inclusion can lead to strong bias in velocity dispersion 
based mass estimates (e.g., Lucey 1983; Borgani et al. 1997; Gen 
1997; Biviano et al. 2006; Wojtak et al. 2007; Mamon et al. 2010). 


To avoid the inclusion of these interloper galaxies, often methods 
use a variety of techniques such as iterative clipping (Yahil & Vi¬ 
dal 1977) or the Gapper technique (Beers et al. 1990; Girardi et al. 
1993) to reach convergence on cluster properties. Alternatively, this 
interloper contamination can be modelled when performing density 
fitting (e.g., Wojtak et al. 2007). 

The final and often deemed most important step of galaxy- 
based techniques is to use properties of the refined membership 
to estimate the cluster mass. One of more traditional methods is 
to apply the virial theorem to the projected phase space distribu¬ 
tion of member galaxies (e.g., Zwicky 1937; Yahil & Vidal 1977; 
Evrard et al. 2008), maintaining the assumption that the cluster is in 
virial equilibrium (and sometimes including the surface term, see 
The & White 1986). Perhaps the simplest of approaches to measure 
the mass is to use richness: the number of galaxies associated with 
the cluster above a certain magnitude limit (e.g., Yee Sc Ellingson 
2003). The distribution of galaxies in projected phase space is also 
used to estimate cluster mass, assuming that the cluster follows a 
Navarro, Frenk and White (NFW) density profile (Navarro et al. 
1996; Navarro et al. 1997). Finally, in the caustic technique, the es¬ 
cape velocity profile is identified in projected phase space through 
an abrupt decrease in phase space density at higher velocities, de¬ 
livering a cluster mass (e.g., Diaferio & Geller 1997; Diaferio 1999; 
Gifford & Miller 2013). 

In our first study (Old et al. 2014, hereafter Paper I), we set 
out to determine the simplest-case baseline by using a clean well 
defined data set based on a Halo Occupation Distribution (HOD), 
hereafter referred to as ‘HODL. This simple model delivers spher¬ 
ically symmetric clusters, idealised substructure, a strong richness 
correlation and isotropic, isothermal Maxwellian velocities. For 
this straight-forward test, we found that, above IO^^Mq, recov¬ 
ered cluster masses are correlated with the true underlying clus¬ 
ter mass with scatter of typically a factor of two. However, below 
IO^'^Mq, the scatter rises and rapidly approaches an order of mag¬ 
nitude. We also found that richness-based approaches produced the 
lowest scatter, though it is not clear if this is due to the simplicity 
of the HODl model used. 

Paper I raised important questions: would a more complex and 
realistic input galaxy catalogue change the performance of the dif¬ 
ferent classes of methods in extracting accurate cluster masses? Is 
the success of the richness-based methods caused by the simplic¬ 
ity of the HOD model used to generate the input galaxy catalogue? 
To address these questions, we test the performance of 25 differ¬ 
ent galaxy-based methods by using two mock galaxy catalogues 
that are produced using more sophisticated, observationally realis¬ 
tic and, most importantly, contrasting models. Using two distinct 
mock catalogues for this test not only allows us to evaluate how, or 
if, our results vary as a result of the model we use, but also allows 
us to explore how different prescriptions of populating galaxies im¬ 
pacts the efficacy of mock galaxy catalogues. The ultimate goal of 
this project is not only to rank cluster mass methods but to gain in¬ 
sight into how we can improve both the cluster mass measurement 
techniques and generate more realistic mock galaxy catalogues. 

The article is organised as follows. We describe the mock 
galaxy catalogue in Section 2, and the mass reconstruction meth¬ 
ods applied to this catalogue are briefly described in Section 3. In 
Section 4, we provide details of our analysis and present our re¬ 
sults on cluster mass and membership comparisons in Section 5. 
We end with a discussion of our results and conclusions in Sec¬ 
tion 6. Throughout the article we adopt a Lambda cold dark matter 
(ACDM) cosmology with Ho = 0.27, Ha = 0.73, as = 0.82 and 
a Hubble constant of Ho = lOOkms”^ Mpc“^ where h = 0.7, 
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although none of the conclusions depend strongly on these param¬ 
eters. 


2 DATA 

This article is the second in a series in which we perform an ex¬ 
tensive comparison of galaxy-based techniques using two differ¬ 
ent mock galaxy catalogues. The two catalogues are produced by 
populating the underlying dark matter simulation with two sophis¬ 
ticated models that, importantly, are fundamentally different in na¬ 
ture. Both the more sophisticated HOD, referred to as ‘HOD2’, and 
Semi-Analytic model, referred to as ‘SAM2’, are described below 
along with a description of how the light cone was constructed. 
To deliver a sample containing both high mass clusters and lower 
mass groups, 1000 groups/clusters are selected from the two di¬ 
verse mock galaxy catalogues by taking the 800 most massive and 
then the next 200 richest clusters. Any duplicate clusters present 
due to the way in which the light cones are constructed as well as 
clusters lying close to the edge of the cone are removed from the 
main analysis leaving, 968 groups/clusters. 


2.1 Underlying dark matter simulation 

We begin by using the Bolshoi dissipationless cosmological simu¬ 
lation which follows the evolution of 2048^ dark matter particles 
of mass 1.35 x 10* from 2 = 80 to z = 0 within a box of 

side length 250 h“^Mpc (Klypin et al. 2011). The force resolution 
of the simulation is 1 kpc and the halo catalogues are complete 
for haloes with circular velocity V^i^c > 50kms“^ (correspond¬ 
ing to Maeop « 1.5 x 10^° The simulation adopts a 

flat ACDM cosmology with the following parameters: Do = 0.27, 
Da = 0.73, (Jg = 0.82, n = 0.95 and h = 0.70 and was run 
with the ART adaptive mesh refinement code. Dark matter haloes, 
substructure and tidal features are identified using ROCKSTAR, 
a 6D FOF group-finder based on adaptive hierarchical refinement 
(Behroozi et al. 2013). This halo finder has been shown to recover 
halo properties with high accuracy and produces consistent results 
with other halo finders (Knebe et al. 2011). The haloes and sub¬ 
haloes found using ROCKSTAR are then joined into hierarchical 
merging trees that describe in detail how structures grow as the uni¬ 
verse evolves. 


2.2 Light cone construction 

The light cones used in this work were produced using the Theo¬ 
retical Astrophysical Observatory (TAO^, Bemyk et al. 2014), an 
online eResearch tool that provides access to semi-analytic galaxy 
formation models and N-body simulations, including tools which 
modify them to produce more realistic mock catalogues. Here, we 
use the light cone generation tool that remaps the original spatial 
and temporal positions of each galaxy in the box onto an observer 
cone specified by the user, which in our case subtends 60° by 60° 
on the sky, covering a redshift range ofO < 2 < 0.15. Note that this 
cone is not flux-limited. However, as in Paper I, we specify a min¬ 
imum r-band luminosity for the galaxies of Mr = -19-1-5 log h 
for both the HOD2 and SAM2 catalogues. 


^ https://tao.asvo.org.au/tao/ 


2.3 Halo Occupation Distribution model 

For the HOD2 model, a galaxy group/cluster catalogue was con¬ 
structed with the halo catalogue using an updated version of the 
model described in Skibba et al. (2006) and Skibba & Sheth (2009). 
We refer the reader to these articles and Paper I for details. Briefly, 
haloes are populated with galaxies whose luminosities and colours 
are modelled such that they approximately reproduce the luminos¬ 
ity function, colour-magnitude distribution, and luminosity- and 
colour-dependant redshift- and real-space clustering in the Sloan 
Digital Sky Survey (SDSS,York et al. 2000). An important assump¬ 
tion in this HOD2 model is that all galaxy properties -their abun¬ 
dances, spatial distributions, velocities, luminosities, and colours- 
are determined by parent halo mass alone, using the mass (M 2 ooc) 
given by the ROCKSTAR algorithm. 

The relevant model updates include the following. First, the 
Skibba & Sheth (2009) model is extended by allowing for a depen¬ 
dence of the colour distribution on halo mass at fixed luminosity 
(More et al. 2011; Hearin & Watson 2013; Rodn'guez-Puebla et al. 
2013), and we include colour gradients within haloes (Hansen et al. 
2009; van den Bosch et al. 2008), which results in red galaxies 
having higher number density concentrations than blue galaxies in 
haloes of a given mass (as measured by Collister & Lahav 2005). 
We include stellar masses based on the Zibetti et al. (2009) calibra¬ 
tion, and the resulting distributions are approximately consistent 
with the Moustakas et al. (2013) stellar mass function. Secondly, 
we update the concentration-mass relation and scatter by adopting 
those of Wojtak & Mamon (2013) and account for the fact that 
galaxies and subhaloes are less concentrated than dark matter (e.g., 
Hansen et al. 2005; Yang et al. 2005b, Wojtak & Mamon 2013) 
by adopting concentration index Cgai = cdm/1.5. Thirdly, the up¬ 
dated model includes a treatment of dynamically unrelaxed sys¬ 
tems, including some non-central brightest halo galaxies, central 
galaxy velocity bias, and massive substructures, all of which de¬ 
pend on host halo mass (see Skibba et al. 2011; Skibba & Maccio 
2011 ). 

With these changes, the motions of galaxies in the haloes are 
no longer isothermal and isotropic, contrary to the HOD model 
used in Paper L For haloes without these effects, the velocity dis¬ 
persion profiles are isothermal and isotropic as in Paper I, with a 
velocity dispersion that depends on halo mass and radius through 
the scaling a^oo ~ \GM 2 oo/R 200 (but see Mamon et al. 2010; 
Munari et al. 2013; Old et al. 2013). The updated model, includ¬ 
ing a more realistic velocity dispersion profile and an anisotropy 
model, will be described in Skibba (in preparation). 


2.4 Semi-analytic model 

The Semi-Analytic Galaxy Evolution (SAGE) galaxy formation 
model used in this work (Croton et al. in preparation) is an updated 
version of that described in Croton et al. (2006). The merger trees 
described in Section 2.1 form the backbone on which the model 
of galaxy formation is applied. Inside each tree and at each red¬ 
shift, virialised dark matter haloes are assumed to attract pristine 
gas from the surrounding environment, from which galaxies form 
and evolve. The model tracks a wide range of galaxy formation 
physics, including reionisation of the inter-galactic medium at early 
times, the infall of this gas into haloes, radiative cooling of hot gas 
and the formation of cooling flows, star formation in the cold disk 
of galaxies and the resulting supernova feedback, black hole growth 
and active galactic nuclei (AGN) feedback through the ‘quasar’ and 
‘radio’ epochs of AGN evolution, metal enrichment of the inter- 
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galactic and intra-cluster medium from star formation, and galaxy 
morphology shaped through secular processes, mergers and merger 
induced starbursts. 

Each group identified by ROCKSTAR has one ‘central’ 
galaxy whose central position and velocity is determined by av¬ 
eraging the positions and velocities of the subset of halo particles. 
Each group also has a number of ‘satellite’ galaxies that maintain 
the positions and velocities of the subhaloes that merged with the 
parent halo. As a result, galaxies retain the ‘memory’ of the dynam¬ 
ical history of the underlying DM simulation (this is not the case 
for the HOD2 model). 

The SAGE model is primarily calibrated against 2 = 0 obser¬ 
vations, including the stellar mass function and SDSS-band lumi¬ 
nosity functions, baryonic Tully-Fisher relation, black hole-bulge 
relation, and metallicity-stellar mass relation. At higher redshift the 
model provides a good match to the star formation rate density evo¬ 
lution and stellar mass function evolution. See Lu et al. (2014) and 
Croton et al. 2014 (in preparation) for more details and focused 
comparisons. 


3 MASS RECONSTRUCTION METHODS 

We present details of the additional cluster mass reconstruction 
methods tested in the second phase of the project and we high¬ 
light below any changes to the methods that participated in Phase I 
of the project. The type of data the methods require as input and a 
summary of the basic properties of all methods are listed in Table 1. 
As for Phase I, we provide a more detailed overview of the meth¬ 
ods in Tables Al and A2 in the appendix. Each method is identified 
by an acronym and the subsection titles for each method are given 
in the form (author name; initial galaxy selection technique, mass 
estimation property). The initial cluster membership is performed 
in three classes: projected phase space, FOE, or Red Sequence. The 
subsequent mass estimation is performed according to five classes 
of methods: richness, projected phase space, radii, velocity disper¬ 
sions, or abundance matching. For detailed descriptions of methods 
that also participated in Phase I of the project, please refer to Pa¬ 
per I. 

3.1 NUM (Mamon; Phase space, Richness) 

In Paper I, NUM was based on the mass derived from a robust linear 
fit to mock clusters analysed by the CLE mass estimation method, 
log(McLE/M 0 ) = a+ 61ogiVf^®^ j 333 ^^^_i, which yielded 
a = 12.02 and b = 1.38. Now NUM uses the robust bilinear fit to 
log (Mcle/Mq) = a + b log 1333 ^^-b clog(l -b z). 

The metric radius is now 1 Mpc comoving. The constants are now 
a = 12.43, b = 1.22, c = -4.25 and a = 12.21, b = 1.24, c = 
—2.53 for the HOD2 and SAM2 catalogues respectively. For a 
given richness logW^^®^ issakms-i’ *■1'® SAM2 masses are typ¬ 
ically 0.19 dex lower than the HOD2 masses. 

3.2 RMl (Rykoff & Rozo; Red Sequence, Richness) 

The Red Sequence Matched-filter Probabilistic Percolation 
(redMaPPer) algorithm (Rykoff et al., 2014), based on the opti¬ 
mised richness estimator A (Rykoff et al., 2012), is a photometric 
cluster finder that identifies galaxy clusters as overdensities of Red 
Sequence galaxies. It has excellent photo -2 performance and A has 
been found to be a low-scatter mass proxy (Rozo & Rykoff, 2014; 
Rykoff et al., 2014). The algorithm is divided into two stages: the 


first is a calibration stage where the Red Sequence model is derived 
directly from the data, and the second is the cluster-finding stage. 
Given a list of cluster/halo positions and estimated redshifts, it is 
also possible to directly compute the richness and photo -2 given 
the Red Sequence calibration. 

As redMaPPer works entirely in observed magnitude space, 
all absolute magnitudes from the input catalog are de-k-corrected 
to observed g and r magnitudes using sdss_kcorrect (Blanton 
& Roweis, 2007). Although redMaPPer can be run using multi¬ 
band data, the current data set comprised only two bands. We use a 
random sample of halo centres as the “seed” spectroscopic galaxies 
used to calibrate the Red Sequence over the redshift range 0.05 < 
2 < 0.15. Only the Phase 2 HOD2 galaxy sample could be used 
with redMaPPer, as the colour model in the Phase 2 SAM2 does 
not result in a prominent enough Red Sequence. 

Two separate runs of redMaPPer are performed, with both us¬ 
ing the same calibration as described above. The first run, denoted 
RMl, directly computed the richness and photo -2 at the location 
of each halo (using the true redshift as a starting point). Mass es¬ 
timates are made using the abundance-matching estimate for A- 
M 200 C, calibrated by Rykoff et al. (2012), which follows a power 
law (see Appendix B of Rykoff et al. 2012). 


3.3 RM2 (Rykoff & Rozo; Red Sequence, Richness) 

This method is similar to RMl, but is a full cluster finding run using 
the algorithm of Rykoff et al. (2014). After detection, the clusters 
are sorted by descending richness, and each cluster is matched to 
the nearest halo within 3a^. Mass estimates are performed as for 
RMl. We note that only for the RM2 run will there be any offsets 
between the redMaPPer cluster centres and the halo centres. 


3.4 SG3 (de Carvalho; Phase space. Velocity dispersion) 

SG3 is a method for the rejection of velocity interlopers to produce 
a final list of cluster members, making no hypotheses about the dy¬ 
namical status of the cluster (e.g. Wojtak et al. 2007). The algorithm 
is similar to the one proposed by Fadda et al. (1996) and used by 
SGI and SG2. It applies the Gapper technique in radial bins with 
sizes of 0.42/i“^Mpc or larger, to guarantee at least 15 galaxies per 
bin. The procedure is repeated until there are no more interlopers 
and the list of members is used to estimate cluster properties. We 
perform virial analysis in an analogous way to Girardi et al. (1998), 
Popesso et al. (2005), Biviano et al. (2006) and Popesso et al. 
(2007). First, we compute the robust aperture velocity dispersion 
((Zap) of the cluster depending on the number of members avail¬ 
able: gapper (< 15) or bi-weight (^15, Beers et al. 1990). Then, 
(Jap is corrected for redshift errors (Danese et al. 1980) and an es¬ 
timate of virial radius is obtained following Girardi et al. (1998). 
These steps lead us to an initial virial mass estimate (equation 5 of 
Girardi et al. 1998), which is then corrected for the surface pressure 
term (The &. White 1986). 

After applying such a correction, i? 200 c is estimated consid¬ 
ering the virial mass density. If My is the virial mass in a volume 
of radius Ra, then i? 200 c = i?A{pv/[200pc(2:)]}^^^'"^, where pv= 
3Mv/(47ri?A) and pc(z) is the critical density at redshift z. Finally, 
assuming an NFW profile, we obtain M 200 C from the interpolation 
(most cases) or extrapolation of the virial mass My from Ra to 
R 200 C ■ This procedure is analogous to what is done by Biviano 
et al. (2006) and Popesso et al. (2007). 
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Table 1. Summary of the cluster mass estimation methods. Listed is an acronym identifying the method, an indication of the main property used to undertake 
member galaxy selection and an indication of the method used to convert this membership list to a mass estimate. The type of observational data required as 
input for each method is listed in the fourth column. Note that acronyms denoted with an asterisk indicate that the method did not use our initial object target 
list but rather matched these locations at the end of their analysis. Please see Tables A1 and A2 in the appendix for more details on each method. 


Method 

Initial Galaxy Selection 

Mass Estimation 

Type of data required 

Reference 

PCN 

Phase space 

Richness 

Spectroscopy 

Pearson et al. (in prep.) 

PEN* 

FOE 

Richness 

Spectroscopy 

Pearson et al. (in prep.) 

NUM 

Phase space 

Richness 

Spectroscopy 

Mamon et al. (in prep.) 

RMl 

Red Sequence 

Richness 

Multi-band photometry, sample of central spectra 

Rykoffetal. (2014) 

RM2* 

Red Sequence 

Richness 

Multi-band photometry, sample of central spectra 

Rykoff et al. (2014) 

ESC 

Phase space 

Phase space 

Spectroscopy 

Gifford & Miller (2013) 

MPO 

Phase space 

Phase space 

Multi-band photometry, spectroscopy 

Mamon et al. (2013) 

MPl 

Phase space 

Phase space 

Spectroscopy 

Mamon et al. (2013) 

RW 

Phase space 

Phase space 

Spectroscopy 

Wojtak et al. (2009) 

TAR* 

FOE 

Phase space 

Spectroscopy 

Tempel et al. (2014) 

PCO 

Phase space 

Radius 

Spectroscopy 

Pearson et al. (in prep.) 

PEO* 

FOE 

Radius 

Spectroscopy 

Pearson et al. (in prep.) 

PCR 

Phase space 

Radius 

Spectroscopy 

Pearson et al. (in prep.) 

PER* 

FOE 

Radius 

Spectroscopy 

Pearson et al. (in prep.) 

MVM* 

FOE 

Abundance matching 

Spectroscopy 

Munoz-Cuartas & Muller (2012) 

ASl 

Red Sequence 

Velocity dispersion 

Spectroscopy 

Saro et al. (2013) 

AS2 

Red Sequence 

Velocity dispersion 

Spectroscopy 

Saro et al. (2013) 

AvL 

Phase space 

Velocity dispersion 

Spectroscopy 

von der Linden et al. (2007) 

CLE 

Phase space 

Velocity dispersion 

Spectroscopy 

Mamon et al. (2013) 

CLN 

Phase space 

Velocity dispersion 

Spectroscopy 

Mamon et al. (2013) 

SGI 

Phase space 

Velocity dispersion 

Spectroscopy 

Sifon et al. (2013) 

SG2 

Phase space 

Velocity dispersion 

Spectroscopy 

Sifon et al. (2013) 

SG3 

Phase space 

Velocity dispersion 

Spectroscopy 

Lopes et al. (2009) 

PCS 

Phase space 

Velocity dispersion 

Spectroscopy 

Pearson et al. (in prep.) 

PES* 

FOE 

Velocity dispersion 

Spectroscopy 

Pearson et al. (in prep.) 


3.5 MVM (Miiller; FOF, Abundance Matching) 

The member galaxy selection stage of MVM has been modified. In 
Phase I, an ellipsoidal boundary was used to define group/cluster 
membership. Now membership is determined by including all 
galaxies within a joint line-of-sight and plane-of-sky distance from 
the group centres until the background galaxy density is reached. 

3.6 CLE & CLN (Mamon; Phase space, Velocity dispersion) 

CLE and CLN are as in Paper I, except that when the group is split 
by the gapper technique, the subsample containing the mean halo 
velocity is kept (instead of the largest one, as in Paper I). 


4 ANALYSIS 

We employ various statistics to examine the performance of the 
mass reconstruction methods including the Root Mean Square 
(RMS) difference between the recovered and input log mass, the 
scatter in the recovered mass, the scatter about the true 

mass and the bias. For the latter three statistics, we assume 

a model where there is a linear relationship between the recovered 
and true log mass and residual offsets in the recovered mass are 
drawn from a normal distribution. Instead of clipping outliers, we 
try the preferable approach of modelling the uncertainties in the 
data as justified in Hogg et al. (2010). We take a Bayesian approach, 
computing a likelihood that is a sum of the probability of obtain¬ 
ing the data point assuming it is drawn from a ‘good’ distribution 
and the probability of obtaining the data point assuming it is drawn 
from a ‘bad’ outlier distribution. This ensures that the measured 
scatter is not affected by a very small number of extreme outliers. 



Figure 1. Example of the marginalised probability distributions produced 
by our MCMC analysis of the PCN method using the HOD2 catalogue for 
the parameters: the slope (m), the intercept (c), the scatter in the recovered 
mass ) and the posterior fraction of data points belong to the ‘bad’ 

outlier distribution (Pb). 

For example, in the case of a method that produces very low scatter 
in general but has, say, one or two extreme outliers, the measured 
scatter will not be falsely inflated. 
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nass for the 25 methods applied to the HOD2 input catalogue. The colour scheme reflects the approach implemented 
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lack line represents the fit to the recovered log mass produced by the MCMC analysis and the filled grey area presents 
1 line represents the 1:1 relation. ‘NR’ in the legend represents the number of missing groups/clusters. The black ticks 
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Each component of this likelihood is weighted by the proba¬ 
bility that any given point belongs to either of these distributions: 

^ = n p* 

i=l,N 

Pi = [(1 - Pb)-P(logMRec,i| logMTrue,i,CriogMR,„,i,m,c) 

“f ^b-P(log A^flTec,* I log A:fxrue,i ) ^outlier 7 trt, c)] .(1) 


5 RESULTS AND DISCUSSION 

Now that we have described our analysis procedure, we move on 
to present the results of the cluster mass estimation comparison. 
We consider an effective method to be one that minimises scat¬ 
ter, has no bias in the amplitude and the slope of the relation be¬ 
tween recovered to true mass and minimises catastrophic outliers 
and missing groups/clusters (whose masses cannot be determined). 
We therefore examine several aspects of method performance in the 
following subsections. 


Here Pb is the posterior fraction of objects belonging to the ‘bad’ 
outlier distribution, i is the variance of the ‘good’ distribu¬ 

tion and m, c are the slope and intercept of the fit respectively, 
which together give the bias at any true or recovered mass. The 
variance of the ‘bad’ outlier distribution is fixed to be a very large 
number with the prior that the variance of the ‘good’ distribution 
must always be smaller than variance of the ‘bad’ distribution. Flat 
priors are adopted for the variance of the ‘good’ distribution, the 
slope, the intercept, while the probability that N data points belong 
to a ‘bad’ outlier distribution must be between zero and one. 

To efficiently sample our parameter space, we utilise Markov 
Chain Monte Carlo (MCMC) techniques that produce posterior 
probability distributions for these parameters. In particular, we use 
the parallel-tempered MCMC sampler EMCEE (Foreman-Mackey 
et ah, 2013). This sampler uses several ensembles of walkers at dif¬ 
ferent temperatures to explore the parameter space. A walker rep¬ 
resents a point in the parameter space and at each iteration of the 
MCMC, the walkers explore by taking a randomly-sized step to¬ 
wards another (randomly chosen) walker i.e., towards another point 
in parameter space. 

Each ensemble of walkers works at a certain ‘temperature’ 
where the likelihood is modified, enabling walkers to easily explore 
different local maxima i.e., preventing walkers becoming stuck at 
regions of local instead of global maxima in the case of a multi¬ 
modal likelihood. We employ 50 walkers at 5 temperatures and 
perform 2800 iterations, including a ‘bum-in’ of 1500 iterations 
that are discarded. In total, 50 x 5 x 2800 = 700 000 points in 
parameter space are sampled for each method and input catalogue. 
We use the autocorrelation length, which is a measure of the num¬ 
ber of evaluations of the posterior required to produce independent 
samples to verify that convergence has been reached. An example 
of the marginalised probability distributions of the parameters pro¬ 
duced by this parallel-tempered MCMC can be found in Figure 1 . 
Figures of the marginalised probability distributions of parameters 
for all methods are available upon request. 

Employing various statistics allows us to examine different 
aspects of the performance of the mass reconstruction methods. 
The RMS encompasses both scatter and bias and, hence, delivers 
the overall uncertainty we can expect for our ensemble of mock 
clusters. The scatter in the recovered mass, , delivers a mea¬ 

sure of the intrinsic scatter, i.e., in the case of no bias (a slope of 
unity) and no normalisation/offset. The scatter about the tme mass, 
(jMTrue > provides a measure of how well a method performs as¬ 
suming there is no normalisation/offset in the relationship between 
recovered and true log mass. Both the scatter in the recovered mass 
and scatter about the true mass are useful quantities to measure 
when comparing methods, assuming one could accurately calibrate 
both the bias and normalisation/offset. The bias at the pivot mass 
is also calculated, where the pivot mass is taken as the median log 
mass of the input groups/clusters sample (log M 200 C,true = 14.05). 


5.1 Scatter in group/cluster mass recovery 

Figure 2 shows the recovered versus input log mass for the case 
of the HOD2 model. The colour scheme reflects the approach 
implemented by each method to deliver a cluster mass from a 
chosen galaxy membership, as introduced in Section 3. These 
colours are magenta (richness), black (phase space), blue (radial), 
green (abundance-matching) and red (velocity dispersion). Meth¬ 
ods which select an initial cluster membership via the FOF linking 
method have square shaped markers, phase-space based methods 
have circle shaped markers and Red Sequence-based methods have 
diamond shaped markers. 

Figure 2 clearly shows that most methods produce significant 
scatter for this HOD2 mock galaxy catalogue. In the case of radial 
based methods, we find an RMS of at least 0.39 dex up to 1.10 dex, 
which translates to a factor of 2.5 and 12.6 respectively. We see a 
better performance of phase-space and velocity dispersion -based 
methods -eps-converted-to.pdffor the HOD2 model where scatter 
is in the range of 0.26 dex up to 0.47 dex, a factor of ~1.8 to 3.0. 
More traditional richness methods based on simply counting the 



0.3 0.4 0.5 

HOD2 RMS (dex) 


Figure 3. RMS cluster mass accuracies with the simple HODl input cat¬ 
alogue versus those derived from the more sophisticated HOD2 catalogue. 
The dotted black line represents a 1:1 relation. Note: PCR lies beyond the 
axes of this figure with an RMS of 1.07 dex for HOD2 and 0.74 dex for 
HODl. 


number of galaxies out-perform almost all other methods based on 
galaxy properties using the HOD2 model. 

Methods PCN, PFN, NUM, RMl and RM2 generate much 
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versus true cluster mass for the 23 methods applied to the SAM2 input catalogue. Same notation as in Figure 2. 
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lower scatter of ~0.18 - 0.26 dex and the abundance matching 
based method, MVM, also performs slightly better than the best 
richness method (NUM), producing a scatter of 0.17 dex. Note that 
according to Poisson statistics and the median number of galaxies 
in both catalogues (31), richness-based methods should produce a 
minimum of scatter of l/(x/3T In 10) = 0.08 dex. This can also 
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0.45 

i 0.4 
-O 

“ 0.35 
cd 

0.3 

0.25 


0.2 


13.6 13.8 14 14.2 14.4 14.6 

log(M200c,Rec /Mq) 

Figure 5. RMS difference in recovered versus true cluster mass versus the 
mean of the recovered log mass distribution, for the 25 methods applied 
to the HOD2 input catalogue. The dotted black line identifies where the 
mean of the true mass distribution lies. The red dashed line represents the 
RMS produced when assuming all clusters have the median true mass of the 
sample. The number next to each methods’ marker represents the number of 
groups/clusters that are not recovered because they are found to have very 
low (< Mq) or zero mass. Note: PCR lies beyond the axes of this 
figure with an RMS of 1.07 dex and log M 200 C = 13.37. 



be compared to the intrinsic scatter of both the HOD2 and SAM2 
number of galaxies versus mass, shown in Figure Bl. The RMS 
scatter in this relation is 0.09 and 0.12 dex, respectively. The recov¬ 
ered log mass distributions for all methods can be seen in Figure E3 
along with the true log mass distributions. Figure 2 also highlights 
the importance of the initial galaxy selection stage of mass esti¬ 
mation. PCR, a method which deduces cluster mass by calculating 
the RMS radius of galaxies within a 1 Mpc aperture and veloc¬ 
ity range, performs poorly without any interloper removal imple¬ 
mented. However, PFR, a method also based on the RMS radius, is 
far less affected by the presence of interloping galaxies as it uses 
galaxies selected via FOF linking. 

As expected, for the majority of methods, the RMS is higher 
than for Paper I, as shown Figure 3, where a catalogue based on 
a simple HOD model was used (HODl). Interestingly, there are 
some methods (abundance matching method MVM, shifting gap- 
per method SG2, and phase space-based methods RW and TAR) 
that actually have lower RMS values for the more complex HOD2 
model. When we examine the residual recovered mass versus true 
mass for the HOD2 catalogue, shown in Figure El, it becomes ev¬ 
ident that the scatter is substantially higher at lower true masses, 
although this effect appears less severe for richness and abundance 
matching based methods. In addition to the RMS, the scatter in the 
recovered mass, the scatter about the hue mass, 
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Figure 6. RMS difference in recovered versus true cluster mass versus the 
mean of the recovered log mass distribution, for the 23 methods applied 
to the SAM2 input catalogue. The dotted black line identifies where the 
mean of the true mass distribution lies. The red dashed line represents the 
RMS produced when assuming all clusters have the median true mass of the 
sample. The number next to each methods’ marker represents the number of 
groups/clusters that are not recovered because they are found to have very 
low (< lO^*' Mq) or zero mass. 
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the slope and the bias at the pivot mass can be seen in Table 2 for 
both the HOD2 and SAM2 models. The final column of the two 
sub-tables shows the merit, a form of ranking based on the RMS. 
It is assigned in different bins: an RMS scatter of below 0.2 dex is 
assigned 8 stars and then decreasing numbers of stars are assigned 
in subsequent bins of size 0.05 dex. The final bin of methods pro¬ 
ducing an RMS scatter greater than 0.5 dex, is given one star. In 
Paper I of the project, the ranking of the methods was not binned 
but instead, each method was assigned a rank between 1 and 23 
corresponding to the lowest and highest RMS. Here, we assign a 
merit according to RMS bins to highlight the similarity/disparity 
between the scatter produced by different methods in a more linear 
fashion. 

We see the same trends in the magnitude of the scatter for 
different classes of methods when we look at the scatter in the re¬ 
covered mass, and the scatter about the true mass, 

According to these values, if we assume that the methods have no 
bias (slope of unity and zero intercept) in the relation between true 
and recovered mass, then they would deliver scatter as low as 0.14 
- 0.15 dex (MVM and NUM respectively). 

Now that we have examined the results for the sophisticated 
HOD2 catalogue, we move on to examine how well the cluster mass 
reconstruction methods perform using the SAM2. Figure 4 shows 
the recovered log mass versus input log mass for 23 participating 
methods. Note that methods RMl and RM2 did not participate as 
the method could not run on the catalogue due to the less prominent 
Red Sequence produced by the SAM2 model. Immediately, we see 
high levels of scatter for almost all methods with the exception of 
NUM. Furthermore, as we saw with the HOD2 catalogue, this scat¬ 
ter appears significantly worse at lower group/cluster masses when 
we look at the residual recovered mass in Figure E2. Exceptions 
are for methods NUM, MPl, TAR and especially PFO, whose scat- 
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Table 2. Mass recoveiy accuracy (RMS, scatter in the recovered mass, cTM^ec’ slope, scatter about the true mass o-A/xrue ranking based on crMT^ue^ 

and the bias at the pivot mass for all clusters, for both the HOD2 and SAM2 input galaxy catalogues. The merit is assigned in different bins according to the 
level of scatter computed by the RMS. A method producing an RMS scatter of below 0.2 dex is assigned 8 stars and then decreasing numbers of stars are 
assigned in subsequent bins of size 0.05 dex. The final bin of methods producing an RMS scatter greater than 0.5 dex, which is given one star. 


Method HOD2 SAM2 



RMS (dex) 


Slope 

'^MTrue 

Bias 

Merit 

RMS (dex) 


Slope 

^ M'Yxxie 

Bias 

Merit 

PCN 

0.26 

0.21 

1.32 

0.16 

0.07 

****** 

0.38 

0.23 

0.78 

0.30 

0.31 


PFN 

0.20 

0.20 

0.89 

0.22 

-0.02 

******* 

0.49 

0.38 

0.79 

0.48 

0.32 

** 

NUM 

0.18 

0.14 

0.83 

0.17 

-0.07 

******** 

0.20 

0.15 

0.45 

0.34 

-0.01 

******* 

RMl 

0.21 
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0.94 

0.19 

0.12 

******* 
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0.19 

0.97 

0.19 

0.11 

******* 
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0.98 

0.36 
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0.35 

0.34 

1.20 

0.29 
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0.27 

0.90 
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****** 
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0.37 

0.29 

1.08 

0.27 

-0.19 


0.31 

0.24 

0.72 

0.32 

-0.18 

***** 

RW 

0.33 

0.31 

1.05 

0.30 

-0.11 

***** 

0.30 

0.29 

0.92 

0.32 

0.05 
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0.27 

0.24 

1.05 

0.23 

-0.12 

****** 

0.31 

0.31 

0.91 

0.34 

-0.03 

***** 
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0.39 

0.34 

1.42 

0.24 

0.10 


0.41 

0.39 

0.93 

0.42 

0.12 
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0.42 

0.34 

1.33 

0.26 

0.15 


0.62 

0.49 

1.01 

0.49 

0.20 
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1.07 

0.79 

1.38 

0.57 

-0.73 
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0.46 

0.76 

0.61 

0.44 
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0.51 

0.38 
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0.66 
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Figure 7. Scatter in the recovered cluster mass versus the slope of the fit to Figure 8. Scatter in the recovered cluster mass versus the slope of the fit 

the recovered log mass, both delivered by the likelihood analysis, for the 25 to the recovered log mass delivered by the likelihood analysis, for the 23 

methods applied to the HOD2 input catalogue. The dotted black line iden- methods applied to the SAM2 input catalogue. This Figure follows the same 

tifies a slope of unity. The number next to each methods’ marker represents notation as in Figure 7. 

the number of groups/clusters that are not recovered because they are found 
to have very low (< 10^^ Mq) or zero mass. 

ter is lower at the low-mass end (though is still comparably large)! 
Not only does this show that the scatter is dependent on the true 
group/cluster mass but it also suggests that this mass dependence is 
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not consistent across all methods. This is confirmed when we ex¬ 
amine the values of scatter produced when the sample is split into 
lower and higher mass groups/clusters, as shown in Tables Cl and 
C2 in the appendix. This implies that if one had prior knowledge of 
whether a sample of objects contained either groups or high mass 
clusters (e.g., from other cluster mass proxies), then more accurate 
masses would be obtained if a method that performed best for that 
mass category were chosen, as opposed to a method with lower 
scatter over the entire mass range. The recovered log mass distribu¬ 
tions for all methods can be seen in Figure E4 along with the true 
log mass distributions. 

One can compare the scatter found for MPO and MPl with the 
scatter found by Mamon et al. (2013) when they tested this method 
on mock projected phase space distributions derived from random 
sampling of the dark matter particle distribution in haloes of hydro- 
dynamical cosmological simulations. Indeed, Mamon et al. (2013) 
found a scatter of 0.040 and 0.058 dex in log i? 200 c for samples 
of 500 and 100 galaxies, which respectively amount to 0.12 and 
0.17 dex for log M 2 ooc. In the present study, limited to the higher 
mass haloes, MPO and MPl achieve 0.28 and 0.23 dex scatter with 
the HOD2 groups/clusters (Table Cl), and 0.22 and 0.21 dex scat¬ 
ter with SAM2 groups/clusters (Table C2). Given that the present 
study uses, on average, lower numbers of galaxies per halo of 38 
(HOD2) and 53 (SAM2) for the high-mass subsamples, our values 
appear consistent with the scatter values of Mamon et al. (2013). 
Similarly, the RW model has been tested by Wojtak et al. (2009), to 
yield 0.13 dex for 300 particle haloes, while in the present study 
limited to high-mass clusters, RW achieves scatter of 0.27 and 
0.26 dex for HOD2 (Table Cl) and SAM2 (Table C2), respectively. 

It is also useful to evaluate the effectiveness of using colours 
to evaluate the virial masses. For example, the methodology used 
in MPl is identical to that of MPO, except that the former is colour¬ 
blind. Table 2 indicates that MPO and MPl have comparable accu¬ 
racies in mass recovery: for the HOD2 and SAM2 samples, MPO 
has a scatter 0.05 and 0.01 dex higher than MPl, but an RMS that 
is 0.03 and 0.09 dex lower. In other words, the colour information 
helps to reduce the bias, but not the scatter. 

5.2 Bias in group/cluster mass recovery 

The methods do not collectively under- or over-estimate the mean 
true mass for the HOD2 groups/clusters, as shown in Figure 5. With 
the exception of radial-based techniques, the methods are clustered 
around the true mean log mass. Interestingly, this agreement was 
not seen in Paper I when the methods systematically underesti¬ 
mated the mean log mass when tested with the more simple HODl 
model. We do not see this underestimation with the more sophis¬ 
ticated HOD2 model as the calibration of velocity dispersions as 
a function of halo mass and radius is updated and the treatment 
galaxy dynamics (which affect the spatial and redshift distribu¬ 
tions) is slightly modified. Figure 5 also indicates that there is no 
strong correlation between the mean recovered mass and scatter 
produced by the methods. 

A measure of the bias at the pivot mass, which reflects the 
bias in the amplitude of the relation between recovered and true 
log mass, is shown in Table 2. For the HOD2 catalogue, it is clear 
that low levels of bias can be produced by many methods: PFN and 
ESC produce a bias of < ±0.03 whilst other methods MPO, MVM 
produce a bias of ^ ±0.05. Within method classes we see a wide 
range of these bias values. 

For the SAM2 catalogue, we also see that the methods do 
not collectively under- or over-estimate the mean true mass for the 


SAM2 groups/clusters, as shown in Figure 6. We do however, see 
slightly larger values in the bias at the pivot mass, although methods 
NUM, ESC, MPO, RW and TAR produce a bias of s: ±0.05. As 
with the HOD catalogue, we see a wide range of these bias values 
within method classes. Figures 7 and 8 and the result of a Spearman 
rank test show that scatter is uncorrelated with slope of the recov¬ 
ered and input mass relation for the HOD2 model, however, the 
scatter is marginally correlated with the slope for the SAM2 (with 
a p-value = 0.0549). Surprisingly, MVM, an abundance matching 
based method with very low scatter in the recovered mass, has a 
low slope of 0.65. This flatter slope artificially boosts the scatter 
about the true mass, as we can see from the values in Table 2. The 
scatter in the recovered mass is 0.14 dex as opposed to for the scat¬ 
ter about the true mass. This suggests that if MVM were able to 
produce a slope equal to unity, the scatter in for this method would 
be as low as 0.14 dex. 

It is important to understand how our results vary due to the 
underlying model used to produce the catalogue. As touched on 
above, we see some differences in the recovered masses for differ¬ 
ent classes of methods using two very different input mock galaxy 
catalogues. Though we see some differences method-to-method, 
collectively, the methods do not systematically have substantially 
higher scatter or more bias in the slope or amplitude for either 
model. 

This is especially clear in Figure 9 and Figure 10 where his¬ 
tograms of the RMS and scatter about the true mass are shown 
for all methods and each model. There is a surprising similarity 
between the RMS and scatter in the recovered mass for both the 
HOD2 and SAM2 models for many methods. This is encouraging, 
as it suggests that either the galaxy population produced by these 
two contrasting models is analogous or the methods are insensitive 
to the differences between these models. 

5.3 Catastrophic outliers and missing clusters 

When we examine the performance of these methods, it is clear 
that there are a number of groups/clusters whose masses are either 
greatly under- or over-estimated. For example, in the HOD2 cata¬ 
logue, there are three clusters with mass greater than 10^® Mq, but 
some methods predict many more clusters with such large masses 
(e.g., PCO (39), ASl (52), AS2 (54) and SGI (42), see Figure E4). 
Obtaining the correct number of high mass clusters is crucial for 
studies selecting high mass clusters - given the steeply falling mass 
function at the high mass end, even a small number of false high 
mass cluster measurements would have a large impact. 

Furthermore, significant underestimations of mass is also very 
detrimental, as cosmological constraining power increases with 
lower mass clusters (as signal to noise increases with decreasing 
mass). For this reason, it is important to assess the fraction of 
groups/clusters for which a method under- or over-estimates the 
mass by a large factor. The percentage of groups/clusters whose 
mass is under- or over-estimated by a factor of 10 is shown for all 
methods and each model in the form of histograms in Figure 11 . 
Groups/clusters whose masses are over-estimated by over a fac¬ 
tor of 10 are shown as a positive percentage and those whose are 
underestimated by over a factor of 10 are shown as a negative per¬ 
centage. The percentage of groups/clusters that are missing i.e., no 
mass was found for these objects, is shown in white. Encouragingly, 
richness-based methods PCN, NUM, RMl, RM2 and abundance- 
based method MVM have extremely low fraction of these failures. 
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PCN PFNNUMRMl RM2 ESC MPO MPl RW TAR PCO PFO PCR PER MVM ASl AS2 AvL CLE CLN SGI SG2 SG3 PCS PFS 

Method 


Figure 9. Histogram of the RMS of the difference between the recovered and true cluster masses (in dex) for all methods applied to the HOD2 (red) and 
SAM2 (blue) input catalogues. The black dashed line represents the RMS produced when we assume all clusters have the same mass. This uniform mass is 
chosen to be the mean input log mass. The number above each bar represents the number of missing groups/clusters. 
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Method 


Figure 10. Histogram of the scatter in the recovered cluster mass (in dex) for all methods using the HOD2 (red) and SAM2 (blue) input catalogues. The 
number above each bar represents the number of missing groups/clusters. 



Method 

Figure 11. Histogram of the percentage of groups/clusters whose recovered mass is over-estimated (positive %) or underestimated (negative %) by a factor of 
ten or more relative to the true mass for all methods applied to the HOD2 (red) and SAM2 (blue) input catalogues. The white segments of each bar represents 
the number of missing groups/clusters. Note that the y-axis of this plot is truncated so that detail at the low percentage range is seen more clearly. PCR falls 
below this truncation with 32.1% of group/cluster masses underestimated by a factor of ten. 
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1 2 3 


Figure 12. Recovered versus true cluster richness for the 25 methods run on the HOD2 input catalogue. The colour scheme reflects the approach implemented 
by each method to deliver a cluster mass from a chosen galaxy membership: magenta (richness), black (phase space), blue (radial), green (abundance-matching) 
and red (velocity dispersion). The solid black line represents the fit to the recovered A^gai produced by the MCMC analysis and the filled grey area presents 
the 3(J boundary of this fit. The red dotted line represents the 1:1 relation. ‘NR’ in the legend represents the number of missing groups/clusters. The black ticks 
that lie across the 1:1 relation represent the minimum and maximum ‘true’ halo A^gai- 
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Figure 13. Recovered versus true cluster richness for the 23 methods run on the SAM2 input catalogue. This Figure follows the same notation as in Figure 12. 
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However, some radial-based and velocity dispersion based 
methods predict masses out by over a factor of 10 for over 50 
groups/clusters (i.e., 5%). This result indicates that if these velocity 
dispersion, radial-hased or phase space -based methods are used, it 
is vital to also apply abundance matching or certain richness based 
techniques as a sanity check to ensure there are no catastrophic fail¬ 
ures that would misrepresent the shape of the mass function. 

The fraction of groups/clusters whose masses are not recov¬ 
ered are shown as white segments in Figure 11. We see a large 
variation between methods, but no correlation of the fraction of 
missing clusters with method class. Methods such as RMl, RM2, 
TAR, PCO, PFO and SG2 do not recover masses for 4 — 8% of 
groups/clusters, whereas many methods recover masses for all clus¬ 
ters e.g., PCN, PFN, NUM, PCR, PFR, MVM, AvL, PCS and PFS. 


5.4 Group/cluster Wgai recovery 

In this section, we present the results of the number of galaxies (i.e., 
the richness) recovered by the cluster mass reconstruction tech¬ 
niques using both the more sophisticated HOD2 and SAM2 input 
galaxy catalogues. Figures 12 and 13 show the recovered log num¬ 
ber of galaxies versus input log number of galaxies for the case 
of the HOD2 model and SAM2 model respectively. The colour 
scheme, lines, symbols and statistics are the same as for the mass 
comparison figures. 
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in the recovered richness, a , the scatter about the true richness, 
(JiVTrue" the slope and the bias at the pivot number of galaxies. 

Again, we see very low RMS values for NUM, MVM and 
radial-based method PCO for the HOD2 catalogue. The outliers 
with higher RMS values for the SAM2 catalogue are PFN, PFO, 
PFR and PFS, methods that select an initial galaxy membership list 
via FOR Red Sequence based methods ASl and AS2, also have 
a very high RMS values, though, this is mostly due to the large 
bias observed at the pivot mass (-0.48 dex). It is evident from Fig¬ 
ure 14 that all methods have lower scatter in the true number of 
galaxies for the HOD2 input catalogue in comparison with for the 
SAM2 input catalogue. This is not unexpected, as it is the nature of 
the HOD2 model to deliver groups/clusters that have a very strong 
mass - richness correlation. Interestingly though, this strong boost 
in scatter for the SAM2 catalogue does not necessarily translate to 
a much larger scatter in the mass, as reflected in Figure 9. 

Now that we have examined the level of scatter for the recov¬ 
ered richness, we move on to look at the bias. From Table Dl, we 
see slopes of < 1 for the SAM2 catalogue but we do not see the 
same behaviour for the HOD2 model and, as in the case of the scat¬ 
ter, this does not translate to a systematic shallower slope for the 
recovered mass. It is important to note that the slope of the Wgai - 
M 200 C relation is lower in the SAM2 as shown in Figure B1 . We 
also find that the recovered richness versus recovered mass is as 
found in Paper I, where the richness-based methods, have, as ex¬ 
pected, very tight relations. In contrast, many other methods have 
more scatter in both recovered number and recovered mass. Note 
that recovering the correct number of galaxies does not necessar¬ 
ily guarantee that the correct member galaxies are being recovered. 
The fact that we see substantially lower scatter in the recovered 
number of galaxies but not the mass, indicates that it is not suffi¬ 
cient to simply obtain the correct number of galaxies. To deliver 
low scatter, it is essential to get the correct membership. 


6 CONCLUSIONS 

We have performed an extensive test of 25 different galaxy-based 
cluster mass reconstruction methods by using two contrasting mock 
galaxy catalogues that are produced using sophisticated, observa- 
tionally realistic HOD2 and SAM2 models, run on the same halo 
merger tree extracted from the same cosmological W-body simu¬ 
lation. The aim of this work is to determine the level of scatter, 
the bias and completeness that these methods produce, giving in¬ 
sight into how we can improve on these techniques while generat¬ 
ing more realistic mock galaxy catalogues. The main results are as 
follows: 


Figure 14. RMS cluster richness errors from the 23 methods applied to the 
SAM2 input catalogue versus those found when applied to the HOD2 input 
catalogue. The dotted black line represents a 1:1 relation. 


In general, we see a stronger correlation of the recovered rich¬ 
ness to the input richness and lower RMS values for the meth¬ 
ods for both the HOD2 and SAM2 catalogues in comparison with 
group/cluster mass. The mean RMS values produced by methods 
using both catalogues are 0.31 dex for mass estimation and 0.21 dex 
for Agai estimation respectively. This is also highlighted in Fig¬ 
ure 14, which shows the RMS difference between the recovered 
and input log Agai for the SAM2 catalogue versus the HOD2 cata¬ 
logue and Table Dl, which shows this RMS, as well as the scatter 


(i) Phase-space and velocity dispersion -based methods deliver 
a similar level of RMS scatter within the range of a factor of ~ 1.8 
- 3, whilst radial-based methods perform significantly worse, de¬ 
livering an RMS scatter of within a factor of ~2.5 - 12. 

(ii) Richness based methods produce a comparably lower level 
of RMS scatter within the range of a factor of ~ 1.5 - 3.1. The lower 
RMS scatter produced by richness based methods for both HOD2 
and SAM2 mock catalogues (where different assumptions are em¬ 
ployed to populate dark matter haloes with galaxies) suggest that 
the good performance of these methods is robust. The abundance 
matching-based technique we tested also produces a comparably 
lower level of RMS scatter within the range of a factor of ~1.5 - 
1.9 for both models. 

(iii) For many, but not all methods, we find that the scatter is 
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group/cluster mass-dependent and the direction of this dependence 
varies across methods. 

(iv) As expected, for the majority of methods, the scatter is 
higher than for Paper I, where 23 methods were tested on a cat¬ 
alogue based on a simple HOD model. Though, interestingly, there 
are four methods that have lower RMS values for the more complex 
HOD2 model. 

(v) We see a large variation of bias in the slope of the recovered 
and input mass relation across all methods for both the HOD2 and 
SAM2 galaxy catalogues. 

(vi) Many methods produce a significant number of catastrophic 
failures, where group/cluster masses are over or under -estimated 
by a factor of ^ 10. For studies selecting high mass clusters, these 
failures can be detrimental due to the steeply falling high-mass end 
of the cluster mass function. For this reason, we recommend that 
richness or abundance matching -based methods are used as a san¬ 
ity check in conjunction with phase-space, velocity dispersion or 
radial -based methods when high cluster masses are recovered. 

(vii) We see a stronger correlation of the recovered to input 
number of galaxies for both catalogues in comparison with re¬ 
covered to input group/cluster mass. The mean RMS produced by 
methods using both catalogues 0.31 dex for mass estimation and 
0.21 dex for Wgai estimation. However, this does not mean the cor¬ 
rect member galaxies are being selected. The boost in scatter from 
the number of galaxies to mass indicates that the selection of the 
correct galaxies (and not just the correct number of galaxies) is a 
key to delivering lower scatter for these methods. 

(viii) We see a variation of bias in the slope of the recovered 
and input number of galaxies relation across all methods for the 
HOD2, however, all methods produce a slope of less than unity for 
the SAM2 galaxy catalogue. 

(ix) Though we see some differences method-to-method, in gen¬ 
eral, methods do not have significantly higher scatter for either the 
more sophisticated HOD2 or the SAM2 galaxy catalogues. This is 
encouraging, as it suggests that either the galaxy population pro¬ 
duced by these two contrasting models is analogous or the methods 
are insensitive to the differences between these models. 

There are several outstanding questions that we hope to ad¬ 
dress in future using our dataset. What is the impact of obser¬ 
vational limitations such as fibre collisions or survey artifacts on 
group/cluster membership and hence mass recovery? What is the 
impact of halo shape and concentration on group/cluster mass re¬ 
covery (Wojtak et al. in preparation)? What produces the catas¬ 
trophic under- or over- estimates in each of the 25 methods? These 
projects share the overall goal of improving or constructing more 
accurate cluster mass reconstruction techniques. 
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APPENDIX A: PROPERTIES OE THE MASS RECONSTRUCTION METHODS 

Table Al. Illustration of the member galaxy selection process for all methods. The second column details how each method selects an initial member galaxy 
sample, while the third column outlines the member galaxy sample refining process. Finally, the fourth column describes how methods treat interloping 
galaxies that are not associated with the clusters. 


Methods 

Member galaxy selection methodology 


Initial Galaxy Selection 

Refine Membership 

Treatment of Interlopers 

PCN Within 5Mpc, 1000 km 

Clipping of ±3 cr, using galaxies within 

Use galaxies at 3 — 5 Mpc to find inter- 


1 Mpc 

loper population to remove 

PFN FOF 

No 

No 

NUMWithin 3 Mpc, 4000 km s“ ^ 

1) Estimate i? 200 c from the relationship be¬ 
tween -R 20 OC and richness deduced from 
CLE; 2) Select galaxies within -R 20 OC and 
withlul < 2 . 7 t 7 NFW(^) 

No 

RMl Red Sequence 

Red Sequence 

Probabilistic 

RM2 Red Sequence 

Red Sequence 

Probabilistic 

Within preliminary i? 200 c estimate 

Gapper technique 

Removed by Gapper technique 

ESC and ±3500 km 

MPO Input from CLN 

1) Calculate R 20 QC ■> Rpi Rred^ ^biue ky 
MAMPOSSt method; 2) Select members 
within radius according to colour 

No 

MPl Input from CLN 

Same as MPO except colour blind 

No 

RW Within 3 Mpc, 4000 kms“^ 

Within -R 20 OC , |24>(_R)|^/^, where R 200 C 
obtained iteratively 

No 

TAR FOF 

No 

No 

PCO Input from PCN 

Input from PCN 

Include interloper contamination in den¬ 
sity fitting 

PFO Input from PFN 

Input from PFN 

No 

PCR Input from PCN 

Input from PCN 

Same as PCN 

PFR Input from PFN 

Input from PFN 

No 

MV^FOF (ellipsoidal search range, cen- 

Increasing mass limits, then FOF, loops until 

No 

tre of most luminous galaxy) 

closure condition 


ASl Within 1 Mpc, 4000 kms“^, con- 

Clipping of ±3 a 

Removed by clipping of ±3 (t 

Strained by colour-magnitude rela¬ 
tion 



AS2 Within 1 Mpc, 4000 km s“ ^, con- 

Clipping of ±3 a 

Removed by clipping of ±3 (t 

strained by colour-magnitude rela¬ 
tion 



AvL Within 2.5 av and 0.8 R 2 Q 0 

Obtain i? 200 c and (Jv by (T-clipping 

No 

CLE Within 3 Mpc, 4000 km s“ ^ 

1) Estimate -R 20 OC from the aperture velocity 

Obvious interlopers are removed by ve- 


dispersion; 2) Select galaxies within -R 20 OC 

locity gap technique, then further treated 


and with |t;| < 2.7(Tj^^^(i?); 3) Iterate 
steps 1 and 2 until convergence 

in iteration by cr clipping 

CLN Input from NUM 

Same as CLE 

Same as CLE 

SGI Within 4000 km 

1) Measure <Jgai, estimate Af 200 c and 

Shifting gapper with minimum bin size 


R 20 OC ; 2) Select galaxies within -R 20 OC ; 3) 

of 250 kpc and 15 galaxies; velocity 


Iterate steps 1 and 2 until convergence 

limit 1000 km s“ ^ from main body 

SG2 Within 4000 km s “ ^ 

1) Measure Ugai, estimate M 200 C and 

Shifting gapper with minimum bin size 


R 20 OC ; 2) Select galaxies within R 20 OC ; 3) 

of 150 kpc and 10 galaxies; velocity 


Iterate steps 1 and 2 until convergence 

limit 500 km s~ ^ from main body 

SG3 Within 2.5 Mpc and 

Measure <Tgai, correct for velocity errors. 

Shifting gapper with minimum bin size 

4000kms“l. Velocity distri- 

then estimate M 200 C and -R 20 OC and apply 

of 420 h~^kpc and 15 galaxies 

bution symmetrized 

the surface pressure term correction 


PCS Input from PCN 

Input from PCN 

Same as PCN 

PFS Input from PFN 

Input from PFN 

No 
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Table A2. Characteristics of the mass reconstruction process of methods used in this comparison. The second, third, fourth and fifth columns illustrate whether 
a method calculates/utilises the velocities, velocity dispersion, radial distance of galaxies from cluster centre, the richness and the projected phase space 
information of galaxies respectively. If a method assumed a mass or number density profile it is indicated in columns six and seven. 


Methods 



Galaxy properties used to obtain group/cluster membership and estimate mass 


Velocities 

Velocity dispersion Radial distance 

Richness 

Projected phase space 

Mass density profile 

Number density profile 

PCN 

Yes 

No 

No 

Yes 

No 

No 

No 

PFN 

Yes 

No 

No 

Yes 

No 

No 

No 

NUM 

No 

No 

No 

Yes 

Yes 

No 

No 

RMl 

No 

No 

Yes 

Yes 

No 

No 

NFW 

RM2 

No 

No 

Yes 

Yes 

No 

No 

NFW 

ESC 

Yes 

Yes 

Yes 

No 

No 

Caustics 

No 

MPO 

Yes 

No 

Yes 

No 

Yes 

NFW 

Yes 

MPl 

Yes 

No 

Yes 

No 

Yes 

NFW 

Yes 

RW 

Yes 

No 

Yes 

No 

Yes 

NFW 

Yes 

TAR 

Yes 

Yes 

Yes 

No 

No 

NFW 

No 

PCO 

Yes 

No 

No 

No 

No 

NFW 

Yes 

PFO 

Yes 

No 

No 

No 

No 

NFW 

Yes 

PCR 

Yes 

No 

Yes 

No 

No 

No 

No 

PFR 

Yes 

No 

Yes 

No 

No 

No 

No 

MVM 

Yes 

Yes 

Yes 

No 

No 
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APPENDIX B: RICHNESS AND VELOCITY DISPERSION - MASS RELATIONS 
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Figure Bl. Upper row: richness versus mass of the 968 groups/clusters for both the HOD2 and SAM2 input catalogues. The intrinsic scatter of the richness 
versus mass relation of the HOD2 and SAM2 catalogue is 0.09 dex and 0.12 dex, respectively. Lower row: velocity dispersion versus mass of the 968 
groups/clusters for both the HOD2 and SAM2 input catalogues. The velocity dispersion is calculated by taking the deviation of the line-of-sight velocities of 
all member galaxies associated with the groups/clusters. The intrinsic scatter of the velocity dispersion versus mass relation is 0.071 dex and 0.066 dex for 
HOD2 and SAM2, respectively. We note that this scatter is higher than Munari et al. (2013), who compute the 3D velocity dispersion, while we consider the 
line-of-sight velocity dispersion, which naturally results in a larger scatter, especially if the clusters are triaxial (see discussion in Mamon et al. (2010). 
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APPENDIX C: MASS RECOVERY ACCURACY EOR LOW AND HIGH GROUP/CLUSTER SAMPLES 


Table Cl. Mass recovery accuracy (RMS, scatter in the recovered mass slope, scatter about the true mass (<^Afxrue^ ranking based on 

cTjVfxrue ^ and high-mass clusters (split according to the median true mass in each subsample) for the HOD2 input catalogue. A method producing 

an RMS scatter of belo\v 0.2 dex is assigned 8 stars and then decreasing numbers of stars are assigned in subsequent bins of size 0.05 dex. The final bin of 
methods producing an RMS scatter greater than 0.5 dex, is given one star 


Method 



HOD2 low masses 





HOD2 high masses 



RMS (dex) 


Slope 

^Mxrue 

Bias 

Merit 

RMS (dex) 


Slope 

•^A/True 

Bias 

Merit 

PCN 

0.23 

0.23 

1.21 

0.19 

0.03 

******* 

0.29 

0.18 

1.28 

0.14 

0.14 

****** 

PFN 

0.21 

0.21 

0.79 

0.27 

-0.01 

******* 

0.19 

0.18 

0.94 

0.19 

-0.04 

******** 

NUM 

0.17 

0.16 

0.72 

0.22 

-0.06 

******=!==!= 

0.19 

0.13 

0.82 

0.16 

-0.09 

******** 

RMl 

0.22 

0.18 

0.57 

0.32 

0.11 

******* 

0.20 

0.17 

0.99 

0.17 

0.11 

******** 

RM2 

0.23 

0.20 

0.55 

0.37 

0.09 


0.20 

0.17 

1.00 

0.17 

0.11 


ESC 

0.44 

0.44 

0.78 

0.57 

-0.02 

^5lS5i< 

0.26 

0.26 

1.16 

0.22 

-0.05 

****** 

MPO 

0.40 

0.39 

1.28 

0.31 

-0.07 

^5lS5i<5i! 

0.30 

0.28 

1.17 

0.24 

-0.01 

***** 

MPl 

0.41 

0.34 

1.07 

0.32 

-0.20 

^5lS5i< 

0.33 

0.23 

1.07 

0.22 

-0.17 

***** 

RW 

0.37 

0.35 

1.24 

0.28 

-0.10 


0.29 

0.27 

1.07 

0.25 

-0.11 

****** 

TAR 

0.31 

0.27 

1.01 

0.27 

-0.14 

***** 

0.23 

0.21 

0.99 

0.21 

-0.10 

******* 

PCO 

0.44 

0.42 

1.48 

0.29 

0.03 

^5lS5i< 

0.35 

0.29 

1.43 

0.20 

0.17 

***** 

PFO 

0.47 

0.40 

1.60 

0.25 

0.12 


0.37 

0.27 

1.31 

0.21 

0.20 


PCR 

1.28 

0.97 

0.93 

1.04 

-0.84 

* 

0.82 

0.56 

0.99 

0.56 

-0.60 

* 

PFR 

0.49 

0.42 

0.45 

0.94 

-0.26 


0.53 

0.34 

0.63 

0.54 

-0.39 

* 

MVM 

0.19 

0.16 

0.59 

0.26 

0.08 

******** 

0.16 

0.13 

0.61 

0.20 

-0.00 

******** 

ASl 

0.50 

0.49 

1.10 

0.45 

0.11 


0.38 

0.37 

1.02 

0.37 

0.08 


AS2 

0.53 

0.49 

0.99 

0.49 

0.21 

* 

0.40 

0.38 

0.92 

0.41 

0.15 


AvL 

0.38 

0.34 

1.09 

0.32 

0.16 


0.29 

0.25 

1.08 

0.23 

0.15 

****** 

CLE 

0.43 

0.42 

1.08 

0.39 

-0.09 

^5lS5i< 

0.33 

0.30 

1.11 

0.27 

-0.14 

***** 

CLN 

0.48 

0.36 

1.27 

0.28 

-0.28 

** 

0.37 

0.25 

1.11 

0.22 

-0.23 


SGI 

0.50 

0.50 

1.11 

0.45 

0.10 

* 

0.35 

0.35 

0.99 

0.35 

0.03 

***** 

SG2 

0.45 

0.36 

0.97 

0.37 

-0.14 


0.32 

0.24 

1.00 

0.24 

-0.18 

***** 

SG3 

0.29 

0.28 

1.08 

0.26 

-0.08 

****** 

0.22 

0.21 

1.05 

0.20 

-0.04 

******* 

PCS 

0.37 

0.33 

1.09 

0.30 

-0.16 

^5lS5i<5i! 

0.31 

0.26 

1.13 

0.23 

-0.18 

***** 

PFS 

0.37 

0.34 

1.42 

0.24 

-0.15 


0.33 

0.27 

1.14 

0.24 

-0.15 

***** 


Table C2. Mass recovery accuracy for low and high mass groups/SAM2 clusters. Same notation as 

Table CL 





Method 



SAM2 low masses 





SAM2 high masses 



RMS (dex) 


Slope 

^A/xrue 

Bias 

Merit 

RMS (dex) 


Slope 

•^AfTrue 

Bias 

Merit 

PCN 

0.41 

0.23 

0.12 

1.96 

0.31 

sfs** 

0.35 

0.22 

0.90 

0.25 

0.27 

***** 

PEN 

0.51 

0.37 

0.26 

1.41 

0.34 


0.47 

0.38 

0.97 

0.40 

0.28 

** 

NUM 

0.19 

0.15 

0.03 

-5.41 

0.03 

******** 

0.22 

0.15 

0.50 

0.30 

-0.10 

******* 

ESC 

0.42 

0.41 

1.13 

0.36 

-0.02 


0.38 

0.26 

1.08 

0.24 

-0.01 


MPO 

0.31 

0.31 

1.01 

0.30 

-0.01 

***** 

0.23 

0.22 

0.89 

0.25 

-0.04 

******* 

MPl 

0.30 

0.26 

0.79 

0.33 

-0.14 

****** 

0.32 

0.21 

0.71 

0.29 

-0.22 

***** 

RW 

0.34 

0.33 

0.89 

0.37 

0.06 

***** 

0.26 

0.26 

0.95 

0.27 

0.04 

****** 

TAR 

0.30 

0.30 

0.37 

0.79 

-0.03 

***** 

0.31 

0.31 

1.09 

0.28 

-0.06 

***** 

PCO 

0.41 

0.38 

0.06 

-6.36 

0.11 


0.41 

0.38 

1.30 

0.29 

0.09 

*2i!5fS 

PEO 

0.60 

0.46 

0.13 

3.53 

0.20 


0.64 

0.57 

1.49 

0.38 

0.13 

* 

PCR 

0.71 

0.54 

0.70 

0.77 

0.45 


0.56 

0.37 

0.60 

0.62 

0.43 

* 

PFR 

0.67 

0.50 

0.37 

1.35 

0.42 


0.57 

0.46 

0.75 

0.60 

0.35 

* 

MVM 

0.33 

0.14 

0.74 

0.19 

0.29 

***** 

0.22 

0.12 

0.55 

0.22 

0.19 

******* 

ASl 

0.63 

0.62 

1.23 

0.51 

-0.08 


0.45 

0.45 

1.30 

0.34 

-0.04 


AS2 

0.62 

0.62 

1.18 

0.52 

0.06 


0.44 

0.44 

1.20 

0.36 

0.07 


AvL 

0.35 

0.30 

1.27 

0.24 

0.19 


0.31 

0.24 

1.12 

0.22 

0.19 

***** 

CLE 

0.33 

0.31 

1.32 

0.24 

-0.11 

***** 

0.28 

0.25 

1.07 

0.24 

-0.11 

****** 

CLN 

0.37 

0.30 

1.14 

0.26 

-0.18 


0.31 

0.24 

1.00 

0.24 

-0.19 

***** 

SGI 

0.46 

0.45 

0.96 

0.47 

0.11 


0.33 

0.32 

0.95 

0.34 

0.08 

***** 

SG2 

0.33 

0.32 

1.26 

0.25 

-0.07 

***** 

0.29 

0.24 

1.01 

0.24 

-0.12 

****** 

SG3 

0.30 

0.21 

0.89 

0.24 

0.21 

***** 

0.26 

0.18 

0.93 

0.19 

0.20 

****** 

PCS 

0.36 

0.31 

1.23 

0.25 

-0.18 


0.28 

0.25 

1.27 

0.20 

-0.14 

****** 

PES 

0.56 

0.49 

0.78 

0.63 

-0.28 


0.55 

0.44 

1.17 

0.38 

-0.31 
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APPENDIX D: RICHNESS RECOVERY 


Table Dl. The RMS, scatter in the observed richness, obs’ slope, scatter about the true richness and the bias at the pivot richness (for the 

HOD2: log Afgai,true = 1-41 and for the SAM2: log Afgai,true = 1-54). 


Method 



HOD2 





SAM2 



RMS (dex) 

^^gal,obs 

Slope 

^•^gal,true 

Bias 

RMS (dex) 

*^-^gal,obs 

Slope 

*^-^gal,true 

Bias 

PCN 

0.12 

0.09 

0.86 

0.10 

0.00 

0.15 

0.12 

0.62 

0.19 

-0.00 

PFN 

0.19 

0.17 

0.92 

0.18 

0.09 

0.44 

0.35 

0.89 

0.39 

0.27 

NUM 

0.11 

0.10 

0.97 

0.10 

0.02 

0.16 

0.15 

0.77 

0.19 

0.04 

RMl 

0.17 

0.14 

0.95 

0.15 

-0.08 






RM2 

0.18 

0.15 

0.98 

0.15 

-0.09 






ESC 

0.14 

0.14 

0.94 

0.14 

0.03 

0.25 

0.20 

0.90 

0.23 

0.03 

MPO 

0.13 

0.12 

1.05 

0.12 

-0.04 

0.20 

0.19 

0.89 

0.21 

-0.07 

MPl 

0.15 

0.13 

1.03 

0.13 

-0.07 

0.23 

0.19 

0.84 

0.23 

-0.11 

RW 

0.13 

0.13 

0.98 

0.13 

0.00 

0.18 

0.18 

0.88 

0.20 

0.02 

TAR 

0.13 

0.13 

0.94 

0.14 

0.01 

0.24 

0.24 

0.90 

0.26 

0.04 

PCO 

0.10 

0.09 

0.85 

0.11 

0.01 

0.14 

0.12 

0.61 

0.19 

0.00 

PFO 

0.19 

0.16 

0.90 

0.18 

0.10 

0.43 

0.32 

0.91 

0.35 

0.29 

PCR 

0.12 

0.09 

0.86 

0.10 

0.00 

0.15 

0.12 

0.62 

0.19 

-0.00 

PFR 

0.19 

0.17 

0.92 

0.18 

0.09 

0.44 

0.35 

0.89 

0.39 

0.27 

MVM 

0.11 

0.09 

0.77 

0.11 

0.02 

0.17 

0.12 

0.63 

0.18 

0.12 

ASl 

0.22 

0.17 

0.78 

0.22 

-0.13 

0.54 

0.15 

0.56 

0.27 

-0.48 

AS2 

0.22 

0.17 

0.78 

0.22 

-0.13 

0.54 

0.15 

0.56 

0.27 

-0.48 

AvL 

0.12 

0.11 

0.97 

0.11 

0.01 

0.17 

0.17 

0.92 

0.19 

0.01 

CLE 

0.15 

0.15 

0.99 

0.15 

0.01 

0.20 

0.20 

0.93 

0.21 

-0.02 

CLN 

0.14 

0.13 

1.07 

0.12 

-0.04 

0.21 

0.19 

0.90 

0.21 

-0.07 

SGI 

0.18 

0.17 

0.94 

0.18 

0.05 

0.23 

0.22 

0.91 

0.25 

0.08 

SG2 

0.17 

0.16 

0.97 

0.16 

-0.04 

0.22 

0.20 

0.92 

0.22 

-0.08 

SG3 

0.21 

0.18 

0.99 

0.18 

0.12 

0.39 

0.16 

0.69 

0.22 

0.37 

PCS 

0.12 

0.09 

0.86 

0.10 

0.00 

0.15 

0.12 

0.62 

0.19 

-0.00 

PFS 

0.19 

0.17 

0.92 

0.18 

0.09 

0.44 

0.35 

0.90 

0.39 

0.27 



22 Old et al. 


APPENDIX E: RESIDUALS AND RECOVERED MASS DISTRIBUTIONS 
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Figure El. Residuals of the recovered versus true cluster mass for the 25 methods using the H0D2 input catalogue. This Figure follows the same notation as 
in Figure 2. 
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Figure E2. Residuals of the recovered versus true cluster mass for the 23 methods applied to the SAM2 input catalogue. This Figure follows the same notation 
as in Figure El. 
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13 15 


Figure E3. Recovered cluster mass distributions for the 25 methods applied to the HOD2 input catalogue. The red dotted line represents the mean of the true 
mass distribution and the grey distributions on each subplot represent the true mass distributions. 
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Figure E4. Recovered mass distributions for the 23 methods applied to the SAM2 input catalogue. This Figure follows the same notation as in Figure E3. 





















































































































